Fluctuation Theorems of Brownian Particles Controlled by a Maxwell's Demon 
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We study the stochastic dynamics of Brownian particles in a heat bath and subject to an active 
feedback control by an external, Maxwell's demon-like agent. The agent uses the information of 
the velocity of a particle and reduces its thermal agitation by applying a force. The entropy of the 
particle and the heat bath as a whole, thus, reduces. Entropy pumping [Phys. Rev. Lett. 93, 120602 
(2004)] quantifies the entropy reduction. We discover that the entropy pumping has a dual role of 
work and heat contributing to free energy changes and entropy production of the open-system with 
the feedback control. Generalized Jarzynski equality and fluctuation theorems for work functional 
and entropy production are developed with the presence of the entropy pumping. 
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Modern nano-technology allows an active control of 
the position and velocity of nano-devices by feedback 
systems. The systems detect the velocity and posi- 
tion of the nano-devices and manipulate a velocity- and 
position-dependent external force applied to the devices. 
We name such controls as position-dependent feedback 
control (PFC) and velocity-dependent feedback control 
(VFC), respectively. The latter has recently been ac- 
complished in reducing the thermal noise of a cantilever 
in atomic force microscopy (AFM) pj and dynamic force 
microscopy ■ In Q , the feedback system detects the ve- 
locity of the cantilever and reduces its thermal vibration 
by actively switching the direction and magnitude of a 
force that controls the cantilever. A VFC experiment has 
also been proposed to control a frictional force acting on 
a small array of particles by limiting the terminal velocity 
of the array with a terminal attractor 0. The physics 
principle of the VFC is to reduce the entropy of nano- 
devices and its surrounding environment, as a whole. The 
control agent can be considered as a Maxwell's demon, 
who gathers informations of nano-devices and use them 
to reduce the entropy. The entropy reduction has also 
been widely studied in the context of engineering control 
theory Q and has long been suggested as a fundamental 
principle of life [f| . 

In 0, as a simple model for the entropy reduction, 
we studied a Brownian particle in heat bath under a 
friction-like force manipulated by a VFC. As the system 
evolves with time, the particle is eventually settled in 
its stationary state. In the stationary state, the average 
kinetic energy of the particle is lower than the surround- 
ing heat bath since the fiction-like external force reduces 
the thermal fluctuations of the particle. Kinetic energy 
is transferred from heat bath to the particle and is ul- 
timately absorbed by the external control agent. The 
entropy of the heat bath, therefore, decreases in the sta- 



tionary state, and the particle and the control agent act 
as a microscopic refrigerator. To quantify the entropy 
reduction, a mesoscopic thermodynamic theory has been 
developed in |fj where we have shown that the entropy 
of the particle and the heat bath changes due to both 
positive entropyproduction and entropy pumping by the 
external agent |y: 



d(S + Sh) dS p 



dS v 



dt 



dt 



dt 



'Electronic address: 
t Electronic address: 



kkimCSu. washington.edu 



qian@amath.washington.edu 



where dS and dSH are entropy change in Brownian par- 
ticles and heat bath, respectively, and dS p is entropy pro- 
duction and dS pu is entropy pumping by a control agent. 
In the case of the friction-like control, dS pu is negative. 
In its stationary state, dS p + dS pu becomes negative. We 
note that the entropy production, dS p , is always positive 
due to the irreversible process of Brownian dynamics and 
this implies the second law of thermodynamics. 

The entropy reduction mechanism by the entropy 
pumping is a unique feature of VFC. Without the VFC 
but only with PFC, the entropy pumping term vanishes 
and the entropy of the particle and the heat bath, 
as a whole, always increases due to the positive en- 
tropy production. In its stationary state, the entropy 
of the particle does not change, so the entropy of the 
heat bath always increases. This implies that in the sta- 
tionary state with PFC alone, heat transfers on aver- 
age from the system to the heat bath due to the fric- 
tional dissipation of work done to the particle by the 
external agent. Therefore, the particle and the control 
agent cannot act as a microscopic refrigerator but as a 
heater without VFC! The Jarzynski equality and fluctu- 
ation theorems for en trop y p roduction and work func- 
tional 0lll!,[li[lJEE0[Il have been studied 
only in systems with the PFC. Thus a novel question 
arises: How are they modified with VFC? The present 
work answers this question. Let us formulate this ques- 
tion by focusing on the Jarzynski equality. The second 
law of thermodynamics is mathematically an inequality: 
the external work done on a system in contact with a 
heat bath, (W), is no smaller than the Helmholtz free 
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energy change of the system, AF. This inequality can be 
quantified through the newly discovered Jarzynski equal- 
ity: {e-? w ) = e~^ AF @, S M M M d, where 
(3 = I/UbTh with Th heat bath temperature. With the 
presence of VFC, however, the equality needs modifica- 
tion due to entropy pumping. Let us consider energy 
balance: (W) = (AH) + (Q), where AH is the inter- 
nal mechanical energy change of the system and Q is the 
heat dissipation from the system to the surrounding heat 
bath. Since the heat bath is in a quasi-static process, 
(W) = (AH)+T H AS H = {AH)+T H (AS p +AS pu -AS), 
with AS P the entropy production, AS pu the entropy 
pumping, and AS the entropy change in the system. 
Then, (W) > (AH)+T H (AS pu -AS) = AF + T„AS pu , 
where AF = (AH) — ThAS is free energy change. Fi- 
nally, we get inequality: (W) — T H AS pu > AF. This im- 
plies that, with the presence of VFC, entropy pumping 
modifies the stochastic work functional in the Jarzynski 
equality. The 2nd law of thermodynamics can be quanti- 
tatively described by entropy production fluctuation the- 
orem 0, H, El and work fluctuation theorem 0, Il2| . 
These theorems are closely related to Jarzynski equality. 
So, they also need to be extended. 

We discover that entropy pumping has a dual role of 
heat and work contributing to free energy change and the 
fluctuation theorems. The former role is already men- 
tioned in the previous paragraph. The latter role can be 
seen from entropy balance: dS p = dS + d(Sn — S pu ). 
The second law of thermodynamics states that the en- 
tropy production dS p is positive. Then, in the stationary 
state, d(Sn — S pu ) > 0. This implies that entropy pump- 
ing modifies the heat bath entropy contributing to the 
fluctuation theorems. 

Langevin Equation - Without losing generality, we con- 
sider one-dimension Brownian dynamics described by the 
following Langevin equation: 



dv 



dH{x,v;a{t)) 
dx 



jv + g(v) + €, 



(i) 



with v the velocity to a particle, 7 the frictional coeffi- 
cient, g(v) a general VFC, and £ Gaussian white noise 
satisfying (£,{t)£,(s)) = S(t — s). H(x, v, ; a(t)) is a time- 
dependent Hamiltonian changing with a parameter a(t) 
varying with time: H(x,v : a(t)) = \v 2 + U(x;a(t)). 
We use unit mass and assume that the Einstein relation 
Th = I/7 holds for the heat bath with Th heat bath 
temperature. The corresponding Fokker-Planck equation 
CP(x,v,t), where 



1 dP(x,v,t) 

becomes — y dt ' 



C = d 2 v - d v [-{d x H{x, v- a(t))} + g(v)] - vd x , (2) 

with d v = d/dv and d x = d/dx. 

Thermodynamics - We first define several terms. An 
internal system is the Brownian particle together with 
the surrounding heat bath. An external system is the 
external agent that manipulates both the control force 
g(v) and internal potential change due to the change of 
a(t). 



We define stochastic heat dQ(t) 13. ITtl fl6| . stochastic 
entropy of the Brownian particle dS(t) and stochas- 
tic entropy pumping dS pu (t) (in the introduction, the 
same notations were used, but hereafter S, S p , and S pu 
are stochastic.) 0: 



dQ(t) = -(-jv t + Z(t))dx t 
dS(t) = -dh\P{x u v u t) 
dS pu {t) = d Vt g(v t )dt. 



(3) 
(4) 
(5) 



These are all stochastic quantities since (xt,vt) has a 
stochastic trajectory. The entropy change in heat bath 
is given due to its isothermal quasi-static nature, 

dS B (t) = /3dQ(t), 

with f3 = 1/Th- The entropy balance is expressed by 



dS + dSn = dS p + dS p 



(G) 



The Eq. © can be considered as the definition of stochas- 
tic entropy production, dS p . Entropy change of the inter- 
nal system is due to not only entropy production but also 
entropy pumping. Finally, energy balance is expressed as 

dH = d t Hdt+d x Hdx+d v Hdv = d t Hdt+gdx-dQ, (7) 

using Eq.([Q). So, it is natural to define work done on the 
particle by external agents, 



dW = d t Hdt + gdx. 



(8) 



Note that all the above stochastic thermodynamic quan- 
tities are defined with Stratonovich prescription, which 
is known to be physically meaningful |l7j . 

Jarzynski equality and fluctuation theorems - The 
Jarzynski equality |3, LLiJ Il4l Il5| and the equality due 
to entropy production fluctuation theorem 0, IE LLJJI can 
be derived by examining the temporal behavior of the 
following quantity: 



f(x, v, t) = (s(x - x t )5(v - v t ) 



x exp 



s=0 



-dSnis) + dSp U (s) + d\nw(x s ,v s , s) 



where w(x, v, s) is an arbitrary weight function. Note 
that (• • •) is a path integral averaging over initial distri- 
bution, P(x, v, 0): 

N 

(■••)= / lim \ \dxidVi(---)P(x N ,v N \x N -i,VN-i) 

J N^oo 

xP(xn-i, vn-i\xn-2,vn-2) - • • P{^i,vi\x ,v )P(x ,v , 0), 

where P(x, v\x',v') is transition probability to find a 
Brownian particle at x, v after a time interval e = t/N 
given x' , v' as initial starting point. —dSn + dS pu can be 
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expressed as j3dH — fldtHdt — (3gdx + d v gdt using Eq. (JTJ , 
©, and ©. Eq.® becomes 



where 
fo(x,v,t) = 
x exp 



f(x,v,t) = w(x,v,t)e 0H( - x ^ a ^f o (x,v,t), 
S(x - x t )S(v - v t ) 



w(x o ,V O ,0)eP H ( x °< v °'' a (o)) 
-t 

—f3(d s H(s)ds + g(v s )dx s ) + d Vs g(v s )ds 



Note that f (x,v,0) = EplM e -0(H(^;a(o))) xhe ti 
derivative of fo(x,v, t) is expressed as 

d t fo(x,v,t) = Cf Q (x,v,t)+f (x 1 v 1 t)[-f3d t H-f3gv+d v g}. 

Its solution becomes fo(x,v,t) = e^^ H< ^ x ' v;a ^ by re- 
quiring w(x,v,0) — P(x 1 v,0). Therefore, f(x,v,t) — 
w(x,v,t). By integrating both the sides of Eq.JHl over x 
and v, we get the following general equalities |lOj . 



exp[-A5H(t) + AS pu (t)]) = 1, (10) 



w(x t ,v t ,t) 
P{x o ,v o ,0) 



where w(x,v,t) is an arbitrary weight function with 
w(x,v,0) = P(x,v,0), and P(x,v,0) is an arbitrary ini- 
tial probability distribution. 

With w(x,v,t) — exp{—/3H(x,v;a(t))]/Z e (t), where 
Z e (t) = J dxdv exp[— f3H(x, v; a(t))], Ea. becomes an 
extended form of Jarzynski-equality: 



/ & -l3W{t)+AS vu {t)\ 



-/3AF(t) 



(11) 



where, using Eq.JSJ), 



W(t) 



~dH(x s ,v a ;a{s)) 



ds 



ds 



is the work done on the particle by external control 
agents, and AF(t) = — In z°(o) ' ^ s ^ ree ener §y ^if- 
ference of two equilibrium state parameterized by a(0) 
and a(t), respectively. Note that the final probability 
distribution does not have to be in equilibrium states 
parameterized by a(t), while the initial one does by pa- 
rameter a(0). 

With w(x,v,t) = P(x,v,t), Ea. (|10|) becomes an ex- 
tended form of an equality related to the entropy pro- 
duction fluctuation theorem |Toj . 

(c^[-AS H (t)-AS(t)+AS pu (t)}) = (eaq>[-A5 p (t)]) = 1. 

(12) 

Eq. mt shows that average entropy production (AS p (t)} 
becomes positive over the finite time interval with or 
without VFC for arbitrary initial distribution, P(x, v, 0). 
Ea. (|12|l also implies that entropy production fluctuation 
theorem holds under VFC with proper definition of S p , 
Eq.©. 



With w(x, v, t) = w[x, v, 0), we obtain a novel equality 

(exp[-A5 ff (t) + AS pu (t)]) = 1, (13) 

over flat initial distribution (P(x,v,0) = constant). 
Without VFC, Eq.(C2J) becomes (exp[-AS ff (i)]) = 1 in- 
dicating that, over the finite time interval, the average 
heat dissipation (ASh) is guaranteed to be positive with- 
out VFC for flat initial distribution only. 

We note that, for different initial probability distribu- 
tions, one can get various equalities while an equality 
related to entropy production fluctuation theorem is in- 
dependent of initial probability distributions. 

Entropy production fluctuation theorem - We now ob- 
tain an extended form of entropy production fluctuation 
theorem: 



P(AS p (t) = a) 



= exp [a] 



P(AS p (t) = -a) 
using the following path integral relation 0, 

P({x s ,v s };a(s)) 



(14) 



P({x t - S , -vt-s}; a{t - s)) 



= exp[AS H {t) - ASpJi)], 



(15) 

where P({x si v s }; a(s)) is the probability to find a path, 
{x s , v s }, with < s < i, starting from xq, v and ending 
at xt, v t and P({xt- Sl — Vt-s}', <x(t — s)) is the probability 
to find a path traced backward. The derivation of Eq. l(T5|) 
is based on the following conditional probability ratio: 



P(x,v\x',v') 



(x, v\e eC \x' , v') 



P(x', —v'\x, —v) (x', —v'\e tC \x, —v) ' 

where C is Fokkcr-Planck operator defined as Eq. J2J . To 
make the transition probability into path integral form, 
we express C into a Weyl-ordered form: 

£ w (x,v,p x ,p v ) = -pi - ivp x - ^ip v [F(x,v) - jv] 
~[F(x,v) - jv]ip v - ~(d v [F(z,v) - jv}^, 



where F(x, v; a(t)) = —d x H(x, v; a(t))+g(v), p x 
and p v = —id v . Then, as e — > 0, 

P(x,v\x',v r ) = J dp x dp v exp[eC w (x,v,p x ,p v ) 

+ip x {x - x 1 ) + ip v (v - v')] 
S(x — x' — ev) 



-id x 



/47re 



x exp 



' F — jv v — v 



'\ 2 



and 



P(x',-v'\x,-v) = 



2c 



S(x — x' — ev) 



x exp 



V47T£ 

F + jv v — v' \' 2 e 
2 2e~ 



[d^F + 1 v)\ 
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where x = (x + x')/2, v = (v + v')/2, and F = F(x, v). 
Therefore, 



P(x,v\x',v') 
P(x', —v'\x, —v) 



exp 



/ _ V — v' \ 
t[F —j-fv-edyF 



= exp[dS H - dS pu }. 



Work Fluctuation Theorem - When the Hamiltonian 
is time-independent, the work fluctuation theorem has 
been obtained [Til Il2| . Like the entropy production 
fluctuation theorem, the work fluctuation theorem is 
extended as follows. From energy balance, AS p (t) = 
-f3AH(t) + p\V(t) + AS(t) - AS pu (t)}, where W(t) = 
Jq dx s g(v s ) in the time-independent Hamiltonian case. 
|A5(i)| does not increase with sufficiently large time t 
on average since AS(t) = — \nP ss (x t , v t ) +lnP(x , v , 0) 
with P ss a stationary distribution as t — > oo, while | (t ) | 
and |A5j,«| increase on average. Therefore, AS p (t) — > 
/3W(t) — AS pu (t) for t — ► oo and extended work fluctua- 
tion theorem holds for t — ► oo: 



P(f3W{t) - AS pu (t) = a) 
P(j3W(t)-AS pu (t)=-a) ' 

The corresponding equality is derived, 



exp [a]. 



lim (exp[-PW(t) + AS pu (t)}) = 1. 



(16) 



(17) 



From Eq.|Q2J|, we find that (W{t) - T H AS pu {t)) > as 
t — > oo with a time-independent Hamiltonian. 

Entropy Pumping - In Q , we have found that entropy 
pumping is related to momentum phase space contraction 
due to g{v) (dS pu /dt = d v g(v)). The extended fluctua- 
tion theorems derived above show a novel role of entropy 
pumping: a dual role of work and heat. The work func- 
tional W in the Jarzynski equality and the work fluctua- 
tion theorem is modified into W — TnAS pu as in Ea. 111(1 
and H16|) . and heat dissipation Q, i.e., entropy of heat 
bath Sh is modified into Sh — S pu as in Eq.(JTUJ|, (|T2")) . 
and (|13|l . The reason of the duality is easy to understand 
from energy balance Eq.Q: AH — W — T h ASr- When 
Sh is modified, W also needs to be modified with AH 
unchanged. 

For definiteness, let us consider and compare three 
examples with U(x:a(t)) = 0: (1) without VFC but 
only with PFC in two and higher dimensional systems 
(g(v) in Eq.JQ) is replaced to g(x).), (2) with friction- 
like VFC, g(v) = —cv with c > 0, and (3) with non- 



friction- like VFC with c < 0. Let the system in a sta- 
tionary state. In the case (1), AS pu = and the av- 
erage work by control force (C) onto Brownian parti- 
cles (BP), (Wc—>bp)> becomes positive from Eq. l|17|l . so 
heat dissipation from BP to heat bath (HB), (Qbp^hb) 
becomes positive. In the case (2), (Wc^bp) < and 
(Qbp-^hb) < 0. However, the modified work and heat 
have opposite signs: (Wc^bp) — Th(AS pu ) > and 
(Qbp^hb)-T„(AS pu ) > 0, where (AS pu ) = -cAt < 
With the modified work and heat, the case (2) be- 
comes the case (1)! In the case (3), (Wc^bp) > 
and (Qbp^hb) > 0. With entropy pumping modifi- 
cation, (Wc^bp) - T H (AS pu ) > and (Qbp^hb) - 
T H {AS pu ) > 0, where (AS pu ) = -cAt > 0. 



Conclusion Remarks - Nano-scale mesoscopic systems 
with VFC are significantly different from the wi dely stud- 
ied overdamped stochastic systems with PFC [l8|. The 
key difference is that the former involves an active en- 
tropy reduction mechanism like a Maxwell's Demon. We 
have recently developed a rigorous mesoscopic thermo- 
dynamic theory for such systems with VFC |(| : from the 
standpoint of the first law of thermodynamics, a friction- 
like VFC makes the heat flow into Brownian particles 
from the heat bath and then be taken out to the external 
control agent in the form of mechanical energy. However, 
from the standpoint of the second law, not all the heat 
can be converted into mechanical energy. Otherwise, we 
would have a perpetual motion machine of the second 
kind. This is the paradoxical part of a Maxwell's De- 
mon. This paradox is resolved by considering the Brow- 
nian particles and demon-like external agent as one whole 
system. Since the Brownian particles cannot be treated 
as one whole system, a question, 'what constitutes the 
stochastic work contributing to the change in free energy 
of the Brownian particles?', arises. In this Letter, we 
have shown that the stochastic work and entropy pro- 
duction appearing in Jarzynski equality and fluctuation 
theorems should include entropy pumping contribution 
due to a dual role of entropy pumping as work and heat. 
With this modification, we arc able to extend all the pre- 
vious results to systems with VFC. 
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